Recovering Structured Signals in Noise: Least-Squares Meets Compressed Sensing

The typical scenario that arises in most “big data” problems is one where the ambient dimension of the signal is very large (e.g. high resolution images, gene expression data from a DNA microarray, social network data, etc.), yet is such that its desired properties lie in some low dimensional structure (sparsity, low-rankness, clusters, etc.). In the modern viewpoint, the goal is to come up with efficient algorithms to reveal these structures and for which, under suitable conditions, one can give theoretical guarantees. We specifically consider the problem of recovering such a structured signal (sparse, low-rank, block-sparse, etc.) from noisy compressed measurements. A general algorithm for such problems, commonly referred to as generalized LASSO, attempts to solve this problem by minimizing a leastsquares cost with an added “structure-inducing” regularization term (`1 norm, nuclear norm, mixed `2/`1 norm, etc.). While the LASSO algorithm has been around for 20 years and has enjoyed great success in practice, there has been relatively little analysis of its performance. In this chapter, we will provide a full performance analysis and compute, in closed form, the mean-square-error of the reconstructed signal. We will highlight some of the mathematical vignettes necessary for the analysis, make connections to noiseless compressed sensing and proximal denoising, and will emphasize the central role of the “statistical dimension” of a structured signal. Christos Thrampoulidis California Institute of Technology, e-mail: [email protected] Samet Oymak California Institute of Technology, e-mail: [email protected] Babak Hassibi California Institute of Technology e-mail: [email protected]